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Response to Arguments 

1 . In view of the Appeal Brief filed on 8/1 5/05, PROSECUTION IS HEREBY 
REOPENED. A new ground of rejection is set forth below. 

To avoid abandonment of the application, appellant must exercise one of the 
following two options: 

(1 ) file a reply under 37 CFR 1.111 (if this Office action is non-final) or a reply 
under 37 CFR 1 .1 1 3 (if this Office action is final); or, 

(2) initiate a new appeal by filing a notice of appeal under 37 CFR 41 .31 followed 
by an appeal brief under 37 CFR 41 .37. The previously paid notice of appeal fee and 
appeal brief fee can be applied to the new appeal. If, however, the appeal fees set forth 
in 37 CFR 41.20 have been increased since they were previously paid, then appellant 
must pay the difference between the increased fees and the amount previously paid. 

A Supervisory Patent Examiner (SPE) has approved of reopening prosecution by 
signing below: 



:xaminer (okl; nas a 



BKAVESH M. MEHTA 
SUPERVISORY PATENT EXAMINER 
TECHNOLOGY CENTER 2600 

Claim Rejections - 35 USC § 103 

2. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 
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3. Claims 1-4, 12-15, 23 and 24 are rejected under under 35 U.S.C. 103(a) as being 
unpatentable over Spight (USPN 4,462,046, previously cited), in view of Corby, Jr. et al. 
(USPN 5,745,387, hereinafter "Corby"). 

4. Regarding claim 1 , Spight discloses a robot system having an image processing 
function for determining orientation, or orientation and position of a robot operation on 
one of a plurality of objects (abstract), the system comprising: a robot 202 (Figure 2); a 
first image capturing device 40 capturing image data of the plurality of objects 
containing respective images of the objects (Figure 1 ; column 3, line 29-38; column 5, 
line 34-48; column 6, line 11-13); Spight further discloses a memory for string reference 
images wherein each desired object to be identified would be defined by a plurality of 
reference signals indicative of a number of particular orientations (capturing direction) of 
the object being viewed (col. 9 lines 4-19, col. 3 lines 29-38). Spight further discloses a 
processor 64 to perform matching (correlation) on the image data containing images of 
the plurality of objects captured by said first image capturing device with each of said 
reference models successively to select one object having an image matched with one 
of said reference models (column 7, line 64-column 8, line 15; column 9, line 59-column 
10, line 11), and to determine orientation, or orientation and position of the robot 
operation based on the image of the selected one object, based on said one reference 
model and the information of its associated capturing direction (column 7, line 28-49; 
column 8, line 16-37; column 11, line 21-37, it is necessary that the orientation of the 
robot must be determined based on the orientation of the object in order to drive the 
robotic effector). 
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Spight does not explicitly disclose that the reference signals are captured using 
the same imaging device used to capture image data of the object, or storing the 
orientation of the robot with respect to the reference object. However it is well known to 
capture images of an environment in known conditions in order to use them as 
reference data for orientation purposes later on. Corby discloses that the use of 
manipulator arms typically requires a method of determining the position and orientation 
of the distal end of the manipulator arm (and/or subparts) (col. 1 lines 44-46). Corby 
further discloses moving the manipulator arm to a particular position and videotaping 
the structure or device which is to be examined (col. 1 lines 56-58). At a later date the 
manipulator arm is positioned at the same site and current data (such as a video image) 
is compared to previous data (col. 1 lines 58-60). 

Spight and Corby are analogous art because they are from the same field of 
endeavor of using image processing to manipulate objects in an environment using a 
robot. 

At the time of the invention it would have been obvious to one of ordinary skill in 
the art to combine Spight and Corby to obtain the reference images using the same 
imaging device . Spight discloses that the images of multiple orientations of the objects 
are compared with reference objects, and Corby discloses the conventional method of 
producing reference objects. The motivation for using the same camera to take the 
reference image and the later images is that it would allow the manipulator arms to 
determine physical changes in the environment. Thus it would have been obvious to 
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one of ordinary skill in the art to combine Corby and Spight to obtain the invention as 
specified in claim 1. 

Claim 12 is similar to claim 1 , thus the same arguments used for claim 1 apply 
equally to claim 12. 

Regarding claims 2 and 13, Spight discloses that said reference models are 
obtained from a part of the image data of the reference object (column 3, line 29-38; 
column 6, line 37-52; column 10, line 50-63). 

Regarding claims 3 and 14, Spight discloses that said reference models are 
obtained by processing the image data of the reference object (column 3, line 29-38; 
column 6, line 37-52; column 10, line 50-63). 

Regarding claims 4 and 15, Spight discloses that said first image capturing 
device 40 comprises a camera for capturing two-dimensional image data (column 5, line 
42-48). 

Claim 23 is similar to claim 1 , thus the same arguments used for claim 1 apply 
equally to claim 23. 

Regarding claim 24, Spight discloses that reference images and reference 
arrangement information is obtained for workpieces/objects of different shapes (plurality 
of configurations of each desired object to be identified), and wherein the finding 
comprises first determining that a reference image of one of the different shapes 
matches the working image of the workpiece, and then finding one reference image of 
the shape that best matches the working image (column 8, line 10-30; column 9, line 4- 
19). 
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5. Claims 5 and 16 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Spight, in view of Corby, Jr. et al M and further in view of Suyama et al. (USPN 
4,879,664, previously cited, hereafter Suyama) or Stauffer (USPN 4,410,804, previously 
cited). 

Spight nor Corby disclose that the image data are captured from a predetermined 
distance. Suyama discloses a three-dimensional position sensor comprising robot- 
teaching apparatus wherein said image data of the reference object are captured by 
said camera (Figure 11a, element 35) from a predetermined distance (column 6, line 
20-37). In addition, Stauffer teaches that if a two-dimensional sensor is used, the image 
processor is unable to simultaneously determine the distance to the object unless the 
objects are always positioned at a known distance (column 1 , line 31 -39). It would have 
been obvious to one of ordinary skill in the art at the time the invention was made to 
capture an image at a predetermined distance in order to adjust the position and 
posture of teaching head 3 with respect to the workpiece 9 as taught by Suyama 
(column 6, line 20-37) or such that the distance to the object need not be determined by 
other means as taught by Stauffer (column 1 , line 31-39). 

6. Claims 6, 7, 11, 17, 18 and 22 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Spight, in view of Corby, Jr. et al., and further in view of Maeno et al. 
(USPN 5,047,714, previously cited, hereafter Maeno). 

Regarding claims 6 and 17, Spight nor Corby disclose that the robot moves a 
second image capture device to have the determined orientation and/or position. 
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Maeno discloses a method of recognizing surface-mounted parts including a robot 
system comprising a second image capturing device (Figure 7c, element 10); wherein 
said robot situates said second image data capturing device to have said determined 
orientation or to have said determined orientation and said determined position with 
respect to the selected one object (column 4, line 44-49), and a processor processes 
second image data captured by said second image capturing device to detect position 
and/or rotational posture of the selected one object with respect to said second image 
data capturing device (column 4, line 54-63). It would have been obvious to one of 
ordinary skill in the art at the time the invention was made to move the second image 
capture device (imaging tube) to have the determined orientation and/or position as 
taught by Maeno in order to locate position detecting patterns located at specific 
positions to accurately and directly determine the position and direction of conductor 
patterns on the printed circuit board (column 1, line 51-59). 

Regarding claims 7 and 18, Spight nor Corby disclose that the robot moves a 
second image capture device to have the determined orientation and/or position. 
Maeno discloses a robot system comprising a second image capturing device for 
obtaining three-dimensional position; wherein said robot situates said second image 
data capturing device to have said determined orientation or to have said determined 
orientation and said determined position with respect to the selected one object, so that 
the second image data capturing device is directed to a characterizing portion of the 
object; and wherein said processor detects the three-dimensional position and/or 
posture of the selected one object based on the position of said characterizing portion 
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obtained by said second image data capturing device (see discussion of claims 6 and 
17 above). Maeno does not explicitly disclose that the three-dimensional position of the 
selected one object, but the examiner takes Official Notice that obtaining three- 
dimensional position of an object is well known in the art. It would have been obvious to 
one of ordinary skill in the art at the time the invention was made to detect the three- 
dimensional position of the object in order to determine the location of the object in 
space so that operations may be carried out upon it. 

Regarding claims 1 1 and 22, Spight discloses that said robot operation is an 
operation of picking up the selected one object from the plurality of objects, some of 
which are overlapped (randomly arranged) with each other (column 6, line 11-13). 

7. Claims 8, 19, 27 and 30 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Spight, in view of Corby, Jr. et al., in view of Maeno and further in 
view of Soderberg (USPN 4,785,528, previously cited). 

Regarding claims 8, 19, 27, and 30, Maeno does not disclose that said first 
image data capturing device is used as said second image data capturing device. 
Soderberg discloses a robotic work positioning system including a camera 34 mounted 
on member 28 of robot 10 which serves as both the first and second image data 
capturing device since it both determines the position of the work, and is moved to have 
the determined orientation and/or position (Figure 1 ; column 2, line 53-column 3, line 1, 
line 44-50; column 5, line 34-41). It would have been obvious to one of ordinary skill in 
the art at the time the invention was made to use said first image data capturing device 
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as said second image data capturing device as taught by Soderberg in order to retrieve 
parts from a conveyor using a vision system to select the desired part from other parts 
and engage the part at precisely the correct location to be properly placed by the robot 
(column 5 f line 60-column 6, line 6). 

8. Claims 25 and 26 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Spight, in view of Corby, Jr. et al., and further in view of Soderberg. 

Regarding claim 25, Spight nor Corby disclose that the robot is used to capture 
the reference images. Soderberg discloses a robot that is used to capture reference 
images, and wherein the reference arrangement information represents arrangements 
of the robot when capturing the reference images (Figure 1 ; column 2, line 53-column 3, 
line 4, line 31-59; column 5, line 60-column 6, line 6). It would have been obvious to 
one of ordinary skill in the art at the time the invention was made to use a robot to 
capture reference images as taught by Soderberg in order to retrieve parts from a 
conveyor using a vision system to select the desired part from other parts and engage 
the part at precisely the correct location to be properly placed by the robot (column 5, 
line 60-column 6, line 6). 

Regarding claim 26, Spight nor Corby disclose that a second imaging device is 
affixed to the robot. Soderberg discloses a camera 34 mounted on member 28 of robot 
10 and is used to determine additional arrangement information used to determine the 
known arrangement of the robot relative to the workpiece (Figure 1 ; column 3, line 44- 
50; column 5, line 34-41 ). It would have been obvious to one of ordinary skill in the art 
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at the time the invention was made to affix an imaging device to the arm of a robot as 
taught by Soderberg in order to retrieve parts from a conveyor using a vision system to 
select the desired part from other parts and engage the part at precisely the correct 
location to be properly placed by the robot (column 5, line 60-column 6, line 6). 

9. Claims 9, 10, 20, 21, 28, 29, 31 and 32 are rejected under 35 U.S.C. 103(a) as 
being unpatentable over Spight, in view of Corby, Jr. et al., in view of Maeno, and 
further in view of Sakakibara et al. (JP 07-270137, previously cited, hereafter 
Sakakibara). 

Regarding claims 9, 20, 28, and 31 , Maeno does not disclose that said second 
image capturing device comprises a three dimensional visual sensor of spot-light 
scanning type. Sakakibara discloses a three dimensional visual sensor usable in robot 
automation (page 1, paragraph 1), of spot-light scanning type capable of measuring 
distance between the sensor and an object (page 3, paragraph 15). It would have been 
obvious to one of ordinary skill in the art at the time the invention was made to utilize a 
three dimensional visual sensor of spot-light scanning type as taught by Sakakibara in 
order to quickly and accurately determine the three dimensional position of an object 
using one device (page 2, paragraph 11). 

Regarding claims 10, 21, 29, and 32, Sakakibara discloses an image data 
capturing device comprising a structured-light unit for irradiating a structured light on the 
selected object and capturing an image of the object including the irradiated light on the 
object (page 3, paragraph 16). 
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Contact Information 



Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to John B. Strege whose telephone number is (571) 272- 
7457. The examiner can normally be reached on Monday-Friday between the hours of 
8-5. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Bhavesh Mehta can be reached on (571) 272-7453. The fax phone number 
for the organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-21 7-91 97 (toll-free). , 
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